We thank all reviewers for their thoughtful feedback, which aided us in sharpening the presentation of our results. 's questions on bounds, we will present them more explicitly in the paper, as briefly described here. We refer R1 to corollary 2.1 Combining this upper bound with the lower bound above (right term in the max), Th2 is also tight w.r.t. 's questions: our contribution focuses solely on expressiveness aspects which draw the boundaries Note that the experiments in fig.1 We are glad for R2's implementation, but since we do not know the experiment details it is hard to Indeed Kaplan et al. employ hyper-parameters tunings (LR, initializations, batch size, etc) as

Zisserman, 2014, He et al., 2016], depth-efficiency was theoretically supported from a variety of

Numerous algorithms are used for nonnegative matrix factorization under the assumption that the matrix is nearly separable. In this paper, we show how to make these algorithms scalable for data matrices that have many more rows than columns, so-called "tall-and-skinny matrices." One key component to these improved methods is an orthogonal matrix transformation that preserves the separability of the NMF problem. Our final methods need to read the data matrix only once and are suitable for streaming, multi-core, and MapReduce architectures. We demonstrate the efficacy of these algorithms on terabyte-sized matrices from scientific computing and bioinformatics.

Numerous algorithms are used for nonnegative matrix factorization under the assumption that the matrix is nearly separable. In this paper, we show how to make these algorithms scalable for data matrices that have many more rows than columns, so-called "tall-and-skinny matrices." One key component to these improved methods is an orthogonal matrix transformation that preserves the separability of the NMF problem. Our final methods need to read the data matrix only once and are suitable for streaming, multi-core, and MapReduce architectures. We demonstrate the efficacy of these algorithms on terabyte-sized matrices from scientific computing and bioinformatics.